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Abstract: Older hospitalized patients are at risk of experiencing adverse events including, but 
not limited to, hospital-acquired pressure ulcers, fall-related injuries, and adverse drug events. 
A significant challenge in monitoring and managing adverse events is lack of readily accessible 
information on their occurrence. 

Purpose: The objective of this retrospective cross-sectional study was to validate diagnostic 
codes for pressure ulcers, fall-related injuries, and adverse drug events found in routinely 
collected administrative hospitalization data. 

Methods: All patients 65 years of age or older discharged between April 1 , 2009 and March 3 1 , 
201 1 from a provincial academic health sciences center in Canada were eligible for inclusion in 
the validation study. For each of the three t5'pes of adverse events, a random sample of 50 patients 
whose records were positive and 50 patients whose records were not positive for an adverse event 
was sought for review in the validation study (n=300 records in total). A structured health record 
review was performed independently by two health care providers with experience in geriatrics, 
both of whom were unaware of the patient's status with respect to adverse event coding. A physician 
reviewed 40 records (20 reviewed by each health care provider) to establish interrater agreement. 
Results: A total of 39 pressure ulcers, 56 fall-related injuries, and 69 adverse drug events were 
identified through health record review. Of these, 34 pressure ulcers, 54 fall-related injuries, and 
47 adverse drug events were also identified in administrative data. Overall, the diagnostic codes 
for adverse events had a sensitivity and specificity exceeding 0.67 (95% confidence interval 
[CI]: 0.56-0.99) and 0.89 (95% CI: 0.72-0.99), respectively 

Conclusion: It is feasible and valid to identify pressure ulcers, fall-related injuries, and adverse drug 
events in older hospitalized patients using routinely collected administrative hospitalization data. 
The information is relatively inexpensive and easy to access with no impact on clinical staff. 
Keywords: geriatrics, patient safety, adverse drug events, pressure ulcers, fall-related injuries 

Introduction 

Every year, millions of older adults (&65 years) receive safe and effective health 
care, yet there is clear evidence that some patients experience unintended harm from 
their care.'"^ Older hospitalized patients have a particularly high risk of experiencing 
adverse events. The elevated risk has been attributed to a number of factors, including 
the increased likelihood of more complicated illness and complex interventions, the 
presence of co-morbid conditions, and higher rates of hospitalization.''^ "' An adverse 
event has been defined as an unintended injury or complication that is caused by 
health care management, rather than by the patient's underlying disease.' Adverse 
drug events, fall-related injuries, and pressure ulcers are examples of adverse events 
that older patients may experience while in hospital.'-^'' 



submit your manuscript | www.dovepress.c 

Dovepress 

http://dx.doi.org/1 0.2 1 47/DHPS.S643S9 



Drug, Healthcare and Patient Safety 20 1 4:6 1 0 1 - 1 08 



101 



© 2014 Ackro/d-Stolan et al. This wotic is published b/ Dove Nedic^ Press Limited, and licensed under Creative Commons Attribution - Non Commercial (unported, v3.0) 
I License. The full terms of the License are available at http://creativecommons.or^/licenses/by-nc/3.0/. Non-commercial uses of the work are permitted without any further 
permission from Dove Medical Press Limited, provided the work is properly attributed. Permissions beyond the scope of the License are administered by Dove Medical Press Limited. Information on 
how to request permission may be found at: http://www.dovepress.com/pennissions.php 



Ackroyd-Stolarz et al 



Doveoress 



Medications are the most common and cost-effective 
therapeutic intervention that older adults will obtain when 
they seek health care.^ It is estimated that more than 80% of 
older adults take at least one medication per day,* and close to 
40% take five or more per day, placing them at an increased 
risk of developing a drug-related problem.''' It has been 
reported that hospitalized older adults receive an average of 
eight or nine different medications during their hospital stay, 
including many "routine" medications, such as sedatives to 
promote sleep.* ' Adverse drug events have been consistently 
identified inpatient safety studies involving acute care. 
An adverse drug event is an injury that is caused by a drug, 
and can include injuries that result if an indicated drug is not 
taken as prescribed. '^-^ Evidence from a report commissioned 
by the American Association of Retired Persons (AARP) to 
synthesize the literature on "medical injury" in older adults 
suggests that drug complications are two-and-a-half times 
more common in older hospitalized patients than in those less 
than 65 years of age.' Certain drugs have also been associated 
with an increased risk of falls in this population.''' 

Unintentional falls comprise a significant proportion of 
injury-related morbidity and mortality in older adults.'^"" 
Approximately one-third of older adults will fall each year; 
of those, one-half will sustain a minor injury and up to 25% 
will sustain a more serious injury.''' '* Fall-related injuries can 
accelerate functional decline and nursing home admission."-'* 
Hospitalization poses unique risks of falling in older patients. 
In addition to the consequences of immobilization, the unfa- 
miliar enviroiment, factors arising from the acute illness, 
and the effects of treatment may all place older hospitalized 
patients at higher risk of a fall.""' Rates cited in the literature 
for in-hospital falls for patients of all ages range from 2.2 to 
7/1,000 bed days in acute care hospitals."'^" However, it has 
been reported that older patients are nine times more likely 
to fall in hospital than younger patients.' 

Pressure ulcers are another common problem in patients 
admitted to hospital. The incidence of pressure ulcers has 
been reported in the range of 0.4% to 3 8%) in acute care, with 
up to 60% of new pressure ulcers developing while a person 
is in hospital. ^'-^^ Older patients are a high-risk group for 
developing pressure ulcers, due in part to age-related changes 
in skin integrity and mobility limitations.'* The presence of 
comorbid conditions such as peripheral vascular disease or 
nutritional deficiencies exacerbates the effects of sustained 
pressure from immobility to increase the risk of developing 
pressure ulcers.^^'^'' Within hours of immobilization, some 
patients are at risk of developing pressure ulcers.'' In addition, 
their presence may adversely affect ftinctional status, which. 



in turn, affects discharge disposition of the patient.^' Thus, 
on an organizational level, some adverse events may have an 
unintended impact on bed availability and patient flow. 

Methods to detect adverse events 

The first step in understanding the impact of adverse events 
and how to prevent them is the development and validation 
of methods to identify when they have occurred. However, 
one of the challenges in a plan to monitor and manage adverse 
events in an organization is the lack of readily accessible 
information on their occurrence. The presence of an adverse 
event may be documented on individual health records, but 
data abstraction from this source is resource intensive. 

Administrative data sources have been used to develop 
patient safety indicators and performance measures, as well 
as to enhance surveillance capability for adverse events.^*"" 
Using a public health approach to the problem, US investi- 
gators from Wisconsin developed and validated surveillance 
criteria for injuries related to medical and surgical care for 
use with administrative data.^* The Wisconsin surveillance 
criteria differ from other measures of adverse events in that 
they focus on harm to a patient as identified by an injury 
coded in the administrative data. Early studies of adverse 
events focused on medical error and negligence.'"" Unlike 
some indicators and performance measures, the Wisconsin 
surveillance criteria are not limited to subsets of hospital- 
ized patients. Their classification scheme uses a combination 
of diagnosis and external-cause-of-injury codes from the 
International Classification of Diseases, Ninth Revision, 
Clinical Modification (ICD-9-CM) to identify adverse events 
or harm to patients. The diagnosis code describes the nature of 
the problem (eg, rash) and the external-cause codes identify 
the mechanism (eg, adverse effects of antibiotics in thera- 
peutic use).^** A validation study examined the performance 
characteristics of the surveillance criteria when compared 
with the "gold standard" of the major epidemiologic studies 
in patient safety - the health record review. The surveillance 
criteria had an overall sensitivity of 59.9% (95% confidence 
interval [CI]: 42.8-75.0) and a specificity of 97.4% (95% CI: 
94.1-98.8).^* The Wisconsin surveillance criteria have not 
been validated using the tenth revision of the International 
Statistical Classification of Diseases and Related Health 
Problems (ICD-IO).'^ 

Use of routinely collected diagnostic data has been 
underutilized as an ongoing decision-support resource 
despite its advantages over other methods for identifying 
adverse events: it is less costly; it captures new information 
on all patients admitted to hospital without having to rely 
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on active reporting by health care professionals; the data 
are readily available in national and international jurisdic- 
tions; and the approach to event-finding can complement 
what is known from other sources and thus provide a more 
comprehensive view of the problem. The need for robust 
monitoring is increasingly important, as the relatively at-risk 
aging population seeking health care is rapidly growing. 
There is a research gap in using administrative diagnostic 
data specifically to inform patient safety improvements in 
the care of older patients. 

Study objective 

The objective of this retrospective cross-sectional study was 
to validate diagnostic codes for pressure ulcers, fall-related 
injuries, and adverse drug events found in routinely collected 
hospitalization data. 

Methods 

Setting 

The validation study was conducted at the Queen Elizabeth II 
Health Sciences Centre (QEII HSC) in Halifax, Nova Scotia, 
Canada. The QEII HSC is the largest teaching hospital and 
academic health sciences center in Atlantic Canada, with 
approximately 1,100 beds and 31,000 admissions per year. 

Subject selection and data source 

Patients (^65 years) discharged between April 1, 2009 and 
March 3 1, 201 1 from the QEII HSC were eligible for inclu- 
sion in the validation study. Patients who spent any time in 
non-acute care during their hospital stay were excluded (eg, 
patients waiting for a nursing home bed) because it was antici- 
pated that these patients would have multiple comorbidities 
and that there would be other factors contributing to the risk 
of adverse events. For patients with more than one hospital- 
ization during the study period, only the first hospitalization 
was included. For each of the three types of adverse events, 
a random sample of 50 patients whose records were posi- 
tive and 50 patients whose records were not positive for an 
adverse event was sought for review in the validation study 
(n=300 records in total). In Canada, the national standards for 
assigning diagnostic codes in administrative data are such that 
conditions are only coded if they have an impact on length 
of stay or management in hospital. Coding standards may 
vary in different countries. 

Adverse events were identified by the diagnostic codes 
selected from ICD, Tenth Revision, Canada (ICD-IO-CA) 
and found in the administrative hospitalization Discharge 
Abstract Database. The diagnostic codes for adverse drug 



events were derived from the Wisconsin surveillance criteria, 
as these have been previously used in our organization and 
they have strong performance characteristics compared with 
health record review.^*-^'' To determine diagnostic codes for 
pressure ulcers and fall-related injuries, the multidisciplinary 
research team systematically examined published sources 
of codes for these adverse events using explicit criteria 
established a priori (eg, yield, availability, coding rules, 
etc) to select candidate codes. ^'"^^-^^ Table S 1 describes the 
list of diagnostic codes used in the validation study. The 
diagnosis type was also examined for those records in which 
adverse events were coded. The diagnosis type is used in the 
Discharge Abstract Database to identify preexisting condi- 
tions present at the time of admission (Type 1 ) and those that 
develop during the hospital admission (Type 2).^' Diagnosis 
types are critical for identifying harm to patients that occurs 
during hospitalization. 

The adverse events identified by the administrative 
Discharge Abstract Database were compared with those 
identified by the gold standard of a structured health record 
review.^ '" "-^^ Two health care providers (a nurse practitioner 
and a pharmacist) experienced in geriatrics independently 
reviewed each patient record for the occurrence of an adverse 
event without any knowledge of the coding in the administra- 
tive data. The reviewers used a standard data abstraction form 
and a structured review process." There was a brief education 
session about the review process and more detailed written 
instructions were provided to ensure that each reviewer went 
through a uniform process. The written instructions included 
items such as confirming the patient name, correct hospital 
admission, and the order in which different parts of the record 
were to be reviewed. Reviewers were asked to identify if there 
was documentation about the presence of a pressure ulcer, 
fall-related injury, and/or adverse drug event. Reviewers were 
asked to note the location in the health record where this infor- 
mation was documented. The form also included a section 
for free text where a reviewer could identify documentation 
that indicated a possible adverse event. The reviewers were 
not asked to assess potential preventability. Using the same 
structured review process, a physician reviewed 20 records 
from each sample reviewed by the health care providers (n=40) 
to establish interrater agreement. The study was reviewed and 
approved by the Capital District Health Authority Research 
Ethics Board. 

Data analysis 

The sensitivity, specificity, and 95% CIs were calculated for 
each type of adverse event. For pressure ulcers and fall-related 
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injuries, diagnostic codes were compared with the results of 
the health record review by the nurse practitioner. For adverse 
drug events, diagnostic codes were compared with results of 
the review by the pharmacist. The interrater agreement was 
calculated for each type of adverse event based on the subset 
of records also reviewed by the physician. 

Results 

The nurse practitioner identified a total of 39 pressure ulcers 
and 56 fall-related injuries in the 185 complete records that 
were reviewed. Of these, 34 pressure ulcers and 54 fall- 
related injuries were identified in administrative data. The 
pharmacist reviewer identified a total of 69 adverse drug 
events in the 99 complete records that were reviewed. Of 
these, 47 were also identified in administrative data. Table 1 
describes the sensitivity and specificity of the diagnostic 
coding for each type of adverse event as compared with the 
expert health record review. The physician reviewed 20 of 
the health records reviewed by each of the other reviewers. 
Interrater agreement was 100% for pressure ulcers and 87.5% 
for fall-related injuries, and ranged from 81.0%-95.2% for 
adverse drug events, depending on whether or not events 
coded as "possible" adverse drug events were included in the 
calculation. Although we randomly selected 300 records for 
review, 15 (5.0% of 300) records were missing one or more 
key documents needed to definitively confirm the presence 
or absence of a documented adverse event (eg, assessment 
note). One of the records reviewed by the nurse practitioner 
and the pharmacist was considered to be incomplete for 
the pharmacist review as it was missing key documents for 
determination of an adverse drug event. 

Five pressure ulcers and two fall-related injuries were iden- 
tified by the nurse practitioner, but were not coded in adminis- 
trative data. Twenty-two adverse drug events were identified 
by the pharmacist, but were not coded in administrative data. 
The adverse events may not have been coded in the administra- 
tive data for reasons related to the coding standards (ie, coding 
restricted to conditions that have an impact on length of stay or 
management in the hospital). This limitation became apparent 



Table I Performance characteristics of diagnostic codes for 
adverse events compared with expert health record reviev/s 



Type of adverse 


Sensitivity 


Specificity 


event 


(95% CI) 


(95% CI) 


Pressure ulcer" 


0.87 (0.72-0.95) 


0.96 (0.80-0.99) 


Fall-related injury" 


0.96 (0.87-0.99) 


0.9! (0.74-0.98) 


Adverse drug event' 


0.68 (0.56-0.79) 


0.90 (0.72-0.97) 



Notes: ^Nurse practitioner review; ''pharmacist review. 
Abbreviation: CI, confidence interval. 



during the case selection phase of the validation study, as there 
were fewer than 50 records coded for in-hospital pressure ulcers 
and fall-related injuries during the first fiscal year. To address 
this limitation, the random selection process was applied to 
hospitalization data for an additional fiscal year (2010-201 1) 
to identify sufficient records with diagnostic codes indicative 
of a pressure ulcer or fall-related injury. 

Discussion 

Valid epidemiologic data are central to the identification of 
risk factors and development of effective prevention strate- 
gies for adverse events. In this validation study, a total of 39 
pressure ulcers, 56 fall-related injuries, and 69 adverse drug 
events were identified through health record review. Of these, 
34 pressure ulcers, 54 fall-related injuries, and 47 adverse 
drug events were also identified in administrative data. Over- 
all, the diagnostic codes for adverse events had a sensitivity 
and specificity exceeding 0.67 (95% CI: 0.56-0.99) and 
0.89 (95% CI: 0.72-0.99), respectively. Different sources 
of information on adverse events provide varying degrees 
of depth and breadth with respect to an adverse event. Use 
of multiple sources of information will provide a more 
complete picture of the nature and extent of adverse events 
in an organization.^* '' 

One of the most common approaches to collecting infor- 
mation about adverse events is through spontaneous reporting 
using an institutional or other centralized incident-reporting 
system. Although reporting provides valuable detail about 
an event, one consistent finding is that adverse events are 
grossly underreported - by as much as 50%-96%.^'''*'*'' The 
biases associated with potential underreporting make it dif- 
ficult to accurately determine the organization-wide impact 
of those adverse events. Accordingly, there are challenges 
with effectively measuring the results of interventions aimed 
at prevention and management of adverse events across an 
entire organization. 

Use of administrative data to support system-level 
monitoring of adverse events has garnered increased atten- 
tion.^*"''-'"'^"' The reported sensitivity and specificity of diag- 
nostic codes in administrative data used to identify adverse 
events is variable. The performance characteristics of the 
diagnostic codes in our validation study compare favorably 
with those of the Wisconsin surveillance criteria. The lat- 
ter have a reported overall sensitivity of 59.9% (95% CI: 
42.8-75.0) and specificity of 97.4% (95% CI: 94. 1-98.8). The 
Wisconsin surveillance criteria include both the diagnosis 
code and external-cause-of-injury codes. The relative utility 
of the codes used in the Wisconsin validation study varied 
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depending on the type of adverse event. The external cause of 
injury codes identified 98.5% of adverse drug events, which 
suggests that this is an important type of code to include. 
Other studies"'-''^ have reported substantially poorer perfor- 
mance characteristics. The variability is partially explained 
by differences in study methodology, case definitions, and 
care setting. A systematic review of ICD-10 codes used to 
identify adverse drug events in administrative data found 
827 ICD-10 codes for adverse drug events that had been 
reported in the literature. Of the 41 papers included in the 
review, only two reported the sensitivity of their codes and 
both involved emergency department data.'*'' For instance, 
Hohl et al used a combination of prospectively collected 
data on adverse drug events linked with administrative data 
from two Canadian emergency departments. Their reported 
sensitivity ranged from 6.8% (95% CI: 4.0-1 1.2) to 28.1% 
(95% CI: 22.3-34.6), depending on the level of certainty that 
the event was attributable to a medication.''^ 

There is scant literature on the performance characteris- 
tics of diagnostic codes in administrative data for identifying 
pressure ulcers and fall-related injuries. Verelst et al validated 
diagnostic codes to screen for five adverse events using the 
Belgian Hospital Discharge Dataset. The adverse events were 
drawn from the Agency for Healthcare Research and Qual- 
ity Patient Safety Indicators and used ICD-9-CM diagnostic 
codes. They determined that the positive predictive value for 
diagnostic codes to identify pressure ulcers ranged from 54.3 
to 74.5 for those that were hospital-acquired and present on 
admission, respectively."^ The positive predictive value is 
considerably higher in our study population (97.1 [95% CI: 
83.3-1.0]). This may be related to the exclusive focus on 
older patients. Not only is this population at higher risk of 
developing a pressure ulcer, it may be that documentation is 
better because of the increased potential for complications 
in older patients. Although this study focused on three com- 
mon types of adverse events in a high-risk population, the 
approach can be applied to patients from all age groups and 
in any acute care facility that collects routine administrative 
diagnostic data. The Wisconsin surveillance criteria have 
been used in pediatric and adult populations. Their criteria 
include validated diagnostic codes for other types of adverse 
events, such as those related to procedures or devices. 

Administrative data have important limitations with 
respect to adverse event surveillance. Administrative data are 
generally collected for purposes other than surveillance or 
research, so the quality and completeness of the data may be 
limited. There is often insufficient clinical detail to perform 
rigorous risk adjustment using administrative data.^**-^' *' 



As described earlier, coding standards are such that conditions 
are only coded if they have contributed to an increased length 
of stay or change in management of care. In addition, coding 
standards typically require a documented physician diag- 
nosis in order to be coded in administrative data. There is 
evidence that physician documentation about some types of 
adverse events (eg, pressure ulcers) is poor.'" '"' This limitation 
became apparent during our validation study, as there were 
insufficient cases of pressure ulcers and fall-related injuries 
identified during 1 fiscal year. In a large validation study of 
patient safety indicators using administrative hospitalization 
data, similar challenges in reaching the target sample size 
for some indicators were reported.'" Fifteen (5.0%) of the 
300 randomly selected records were missing one or more key 
documents. These limitations in documentation underscore 
the importance of using multiple sources of data on adverse 
events to understand the full scope of the problem.-" *" The 
use of diagnostic data from administrative sources will likely 
result in an underestimate of the number of adverse events, yet 
it is anticipated that it will yield important information using 
fewer resources than other methods of event-finding. 

Use of administrative data for monitoring permits 
population-based analyses that can be used to identify risk 
factors, impact on hospital length of stay, disposition, and 
costs associated with adverse events. Administrative data 
also provide a feasible, relatively low-cost mechanism for 
evaluating the impact of prevention strategies on these 
same measures. These attributes offer important tools for 
organization-level monitoring that will augment data col- 
lected from other sources, such as incident reporting systems. 
Once high-risk patients and/or settings are identified using 
administrative data, more detailed methods such as quality 
reviews or case studies can be used to understand the con- 
tributing factors and system-, patient-, and/or provider-level 
issues so that prevention strategies can be appropriately tai- 
lored. As with all approaches to monitoring adverse events, 
an essential element includes one or more mechanisms by 
which to broadly share the information to maximize learning 
and the potential for making safety improvements to reduce 
harm to patients. 

Conclusion 

Patient safety has become a critical focus for health care organi- 
zations and accreditation bodies in the last decade. Prevention 
of adverse events is very relevant to a growing number of older 
patients seeking care, many of whom are especially vulner- 
able to the unintended harm that may occur in health care. 
Fall-related injuries, pressure ulcers, and adverse drug events 
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are common and costly examples of potentially preventable 
adverse events in older hospitalized patients. The results of 
this validation study demonstrate that it is feasible and valid to 
identify adverse events in hospitahzed patients using routinely 
collected administrative data. Although the approach likely 
underestimates the extent of the problem, the information is 
relatively inexpensive and easy to access, with no impact on 
clinical staiJ. Administrative data can provide a valuable tool 
for ongoing monitoring of the incidence and impact of adverse 
events as well as the identification of potentially modifiable 
risk factors upon which targeted prevention strategies can be 
developed and evaluated. 
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Supplementary material 

Table SI Summary of ICD-IO-CA codes for adverse events 
Adverse drug events 

G04.0, G2I.0, G62.0, L2S.I, L27.0, L27.I, O3S.S0I, O3S.S03, O3S.S09, 
P04.4, T36.0-T37.5, T37.8-T39.4, T39.8-T40.0, T40.2-T40.6, 
T4 1 .0-T4 1 .5, T42.0-T42.8, T43.0-T43.6, T43.8-43.9, T44.0-T48.7, 
T49.0-TS0.9, T78.8, T88. 1-T88.3, T88.S, T88.7, T80.3-T80.6, 
T80.8-T80.9, T96.0, X40-X43, X60, X62-X64, X8S, YI0-YI4, 
Y40.I-Y4I.5, Y4I.8-Y43.6, Y43.8-Y44.7, Y44.9, Y4S.0-Y4S.S, 
Y45.8-Y46.8, Y47.0-Y47.S, Y47.8-Y48.5, Y49.0-Y49.S, Y49.7-Y50.I, 
YS0.8-YSS.7, YS6.0-YS8.6, YS8.8-YS9.0, YS9.2, YS9.3, YS9.8, YS9.9, 
Y63.0, Y64.0, and Y6S.0.' 

Pressure ulcers 

L89.0-L89.8." 

Falls 

W0I-W08, WIO, and WI7-WI9 - cases selected on the basis of 
diagnosis type (post-admit; Type 2) and place of occurrence codes 
(school/other institutional and public area; U98.2).'''' 
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